Focus adjustment apparatus and method for controlling the same

ABSTRACT

A focus adjustment apparatus includes an object detection unit configured to detect feature information of an object, a plurality of signal generation units configured to accumulate charges to generate and output a pair of image signals respectively, and a focus detection unit configured to control charge storage of the plurality of signal generation units and detect a focus based on a phase difference between a pair of image signals output from a first signal generation unit corresponding to the feature information of the object detected by the object detection unit, wherein if reliability of detection by the object detection unit is a first level which is higher than a second level, the focus detection unit sets charge storage time of the first signal generation unit longer than that in a case of the second level.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a focus adjustment apparatus thatperforms autofocus (AF) control by performing tracking according tofeature information of an object.

2. Description of the Related Art

There has conventionally been discussed a technique for detectingfeature information such as a face or a color and executing focusingcontrol based on a range-finding result of a phase difference autofocus(AF) sensor corresponding to a position of the feature information.Japanese Patent Application Laid-Open No. 2010-186004 discuses atechnique for performing focus adjustment by phase difference AF bytracking a face area.

Japanese Patent No. 3442426 discusses a technique for storing in advancea relationship between an output of a photometric sensor and an outputof an AF sensor as a setting value, and adjusting charge storage time ofthe AF sensor based on high and low of luminance.

However, because of a principle of the phase difference AF and theface/color detection, features of an object each suited for detectionare incompatible to each other. More specifically, for the phasedifference AF, high focusing accuracy is acquired in the case of anobject having contrast. On the other hand, for the face/color detection,detection of an object is easier in an area having a uniform color suchas a skin color for which the phase difference AF is not suited. Theseopposing features cause an issue that sufficient focusing accuracycannot be acquired if focusing control is executed in a position where aface or a color has been detected. In the technique discussed inJapanese Patent No. 3442426, the charge storage time of the AF sensor ischanged based on the luminance information of the object. Inconsequence, an issue, namely, reduction in accuracy of the phasedifference AF caused by low contrast when AF is executed by using thefeature information such as a face or a color, cannot be avoided.

SUMMARY OF THE INVENTION

An example of the present invention is directed to improvement of phasedifference autofocus performance in a tracked area while maintainingtracking accuracy using feature information such as a face or a color.

According to an aspect of the present invention, a focus adjustmentapparatus includes an object detection unit configured to detect featureinformation of an object, a plurality of signal generation unitsconfigured to accumulate charges to generate and output a pair of imagesignals respectively, and a focus detection unit configured to controlcharge storage of the plurality of signal generation units and detect afocus based on a phase difference between a pair of image signals outputfrom a first signal generation unit corresponding to the featureinformation of the object detected by the object detection unit, whereinif reliability of detection by the object detection unit is a firstlevel which is higher than a second level, the focus detection unit setscharge storage time of the first signal generation unit longer than thatin a case of the second level.

According to another aspect of the present invention, a method forcontrolling a focus adjustment apparatus including a plurality of signalgeneration units configured to accumulate charges to generate and outputa pair of image signals respectively includes detecting featureinformation of an object, and controlling charge storage of theplurality of signal generation units and detecting a focus based on aphase difference between a pair of image signals output from a firstsignal generation unit corresponding to the detected feature informationof the object, wherein if reliability of detection of the featureinformation of the object is a first level which is higher than a secondlevel, charge storage time of the first signal generation unit is setlonger than that in a case of the second level.

Further features and aspects of the present invention will becomeapparent from the following detailed description of exemplaryembodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the invention.

FIG. 1 illustrates a configuration of a camera according to a firstexemplary embodiment.

FIG. 2 is a block diagram illustrating a camera function according tothe first exemplary embodiment.

FIGS. 3A to 3F illustrate correspondence between an autofocus (AF)sensor surface and an autoexposure (AE) sensor surface according to thefirst exemplary embodiment.

FIG. 4 (including FIGS. 4A and 4B) is a flowchart illustratingprocessing performed by an AF processing unit according to the firstexemplary embodiment.

FIG. 5 is a flowchart illustrating processing performed by an AE imageprocessing unit according to the first exemplary embodiment.

FIG. 6 is a flowchart illustrating tracking range-finding point adoptiondetermination processing according to the first exemplary embodiment.

FIG. 7 is a flowchart illustrating AF charge storage extensiondetermination processing of a tracking range-finding point according tothe first exemplary embodiment.

FIG. 8 is a flowchart illustrating AF charge storage extensiondetermination processing of a tracking range-finding point according toa second exemplary embodiment.

FIG. 9 is a flowchart illustrating AF charge storage completion waitingprocessing of a tracking range-finding point.

FIGS. 10A and 10B illustrate search for a tracking area on an AE sensorsurface.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the inventionwill be described in detail below with reference to the drawings.

FIG. 1 is a block diagram illustrating a configuration example of adigital camera (hereinbelow, simply referred to as “camera”) 100 as anexample of an imaging apparatus according to the present invention. FIG.2 is a configuration diagram illustrates a camera function according toan example of the present invention. A lens unit 202 is detachablyattached to a camera body 201 via a mount. In the present exemplaryembodiment, the lens unit is not described in detail. A lens microprocessing unit (MPU) in the lens unit 202 controls driving of a lensdriving unit 221, a diaphragm control unit 220, and so on based on aninstruction from a camera MPU 6.

The camera MPU 6 includes a read-only memory (ROM) that stores a programfor controlling a camera operation, a random access memory (RAM) thatstores variables, and an electrically erasable programmable read-onlymemory (EEPROM) that stores various parameters. The camera MPU 6executes the program stored in the ROM to realize operations of thecamera body 201 including focus detection processing described below.

The camera MPU 6 acquires luminance information of an object from aphotometric/object feature information detection unit 10, and executesfocus detection by a focus detection unit 7. According to the presentexemplary embodiment, the photometric/object feature informationdetection unit 10 has a function of detecting feature information suchas a face or a color of an object. For an area corresponding to colorinformation detected by the photometric/object feature informationdetection unit 10, the camera MPU 6 calculates a defocus amount bydriving the focus detection unit 7.

FIGS. 3A to 3F illustrate correspondence between an AF sensor surfaceand an AE sensor surface according to the present exemplary embodiment.FIG. 3B illustrates the AE sensor surface. According to the presentexemplary embodiment, an AE sensor 211 included in thephotometric/object feature information detection unit 10 includes aplurality of pixels 301 for detecting red, green, and blue (RGB)components illustrated in FIG. 3B. FIG. 3A illustrates an AF sensorsurface 302. An AF sensor (signal generation unit) included in the focusdetection unit 7 includes a plurality of line sensor pairs 303 andrange-finding points 304 illustrated in FIG. 3A. Each line sensorincludes a plurality of pixels.

The focus detection unit 7 executes AF control based on a phasedifference between a pair of image signals output from the line sensorpair 303 corresponding to the respective range-finding points 304. Inother words, the focus detection unit 7 executes the AF control bydetecting a relative position shifting amount in a division direction ofa light flux from the object. All the areas of the plurality of pixels301 in the AF sensor 211 are set in a positional relationshipcorresponding to the AF sensor surface 302.

In place of the AF sensor of the exemplary embodiment, imaging planephase difference AF can be performed in which focus detection by a phasedifference method is executed using an output from a pixel on an imagesensor. More specifically, there is a method for using an output from anAF pixel, which receives a light flux in which a part of an exit pupilof a photographic optical system is shielded. There is also a method fordividing a pixel below one microlens and using an output from eachdivided pixel. In addition, the phase difference AF using the AF sensorand the imaging plane phase difference AF can be combined.

The focus detection unit 7 designates maximum permissible storage timeto the AF sensor. When the maximum permissible storage time has elapsed,or sufficient charges of the AF sensor have been stored, the focusdetection unit 7 notifies the camera MPU 6 of reading completion. Forexample, when it is determined that the sufficient charges are stored inthe center AF sensor, reading of other AF sensors can also be completed.For an operation of the focus detection unit 7, a technique concerning aknown phase difference AF sensor is employed. As a camera lens system,the camera MPU 6 executes AF control by repeating data exchangecommunication with the lens, AF processing, and lens drivingcommunication.

A shutter driving unit 8 drives a shutter (not illustrated) according toan instruction from the camera MPU 6. A dial unit 9 is an operation unitthat enables a user to perform various settings, such as a continuousshooting speed, a shutter speed, an aperture value, and an imaging modeon the camera body 201. An imaging plane 11 is a light receiving surfaceof an image sensor 208 and receives an object image when a main mirror(and a sub-mirror) is moved out of an optical path to open the shutter.A display unit 12 includes a display apparatus, such as a liquid crystaldisplay (LCD) or an organic electroluminescent (EL) display, andexecutes review display of an image captured by the imaging plane orgraphical user interface (GUI) display, such as a menu screen.

Next, referring to a configuration diagram in FIG. 2, a function of theimaging apparatus according to the present exemplary embodiment will bedescribed. An operation detection unit 213 detects an operation executedby the user via the dial unit 9 or a button, a switch, or a connectiondevice (not illustrated) attached to the camera body 201, and transmitsa signal according to the operation content to a system control unit 203in the camera MPU 6. A mirror control unit 212 executes mirror controlbased on a control signal transmitted from the system control unit 203.

The system control unit 203 reads charge storage data from the linesensor pair 303 included in the AF processing unit 205 of the focusdetection unit 7 when the operation detection unit 213 detects a switch(SW) operation or in a mirror-down state during continuous shooting. Thesystem control unit 203 executes, based on the read data, selection of arange-finding point for focus adjustment and focus adjustmentcalculation. Then, the system control unit 203 transmits a lens drivingsignal based on the calculation result to the lens driving unit 221 viathe lens MPU. The lens driving unit 221 moves the lens based on thesignal transmitted from the system control unit 203 to execute focusingcontrol.

The image sensor 208 photoelectrically converts a light flux entered viathe lens into an electric signal to generate image data. The displaycontrol unit 209 causes the display unit 12 to display an image of animaging result. A main memory 210 is a storage device for storing datanecessary for calculation executed by the system control unit 203 and anAE image processing unit 204.

The AE image processing unit 204 performs various control operationsbased on the image data read out from the AE sensor 211. An AE sensorsurface has a multi-division sensor structure such as that of the pixel301 illustrated in FIG. 3B. The AE sensor includes R, G, and B filtersand can detect face or color information as object feature information.A detection result of the face or color information of the object istransmitted to the AF processing unit 205 for each photographingoperation. For example, the detection result of the face information canbe transmitted once for several photographing operations, and thedetection result of the color information can be transmitted to the AFprocessing unit 205 for each individual photographing operation. One ofother processes performed by the AE image processing unit 204 isautomatic exposure calculation, and exposure calculation processing isperformed based on an output result of the AE sensor 211. According tothe present exemplary embodiment, the system control unit 203 switchescontrol of the AF processing unit 205 according to the detection resultand the detection area of the face or color information of the object.The processing will be described in detail below.

Next, referring to FIGS. 3A to 3F, the AF sensor surface and the AEsensor surface of the present exemplary embodiment will be described.The focus detection unit 7 includes the AF sensor surface 302 includinga plurality of line sensor pairs 303. On the AF sensor surface 302, aplurality of range-finding points 304 is disposed to be selectable bythe user. In each range-finding point, a corresponding AF line sensor303 is disposed. FIG. 3A illustrates an example where 61 selectablerange-finding points are arranged. At the range-finding point 304, apair of line sensors respectively for vertical detection and horizontaldetection is disposed.

The photometric/object feature information detection unit 10 includes anAE sensor surface. As illustrated in FIG. 3B, the AE sensor surface isdivided into pixels 301 as a plurality of photometric areas, andphotometric processing can be performed on an area corresponding to arange-finding point of the AF sensor.

For example, FIGS. 3C and 3D illustrate a positional relationshipbetween a detection state of the AE sensor surface (illustrated in FIG.3D) and the AF sensor surface (illustrated in FIG. 3C) when an object(person) 305 is detected on the AE sensor surface. Upon detecting aposition or a size of a face of the object 305 on the AE sensor surface,the photometric/object feature information detection unit 10 notifiesthe system control unit 203 of detected information. The system controlunit 203 generates, based on the received information, range-findingpoint information on the AF sensor surface 302 corresponding to an areaof the detected face. In the illustrated case, an area 306 is arange-finding point area.

FIGS. 3E and 3F illustrate a positional relationship between a detectionstate of the AE sensor surface (illustrated in FIG. 3F) and the AFsensor (illustrated in FIG. 3E) when a relatively large object (person)is detected. In this case, the photometric/object feature informationdetection unit 10 detects an area 307 corresponding to eyes of theperson. The system control unit 203 identifies an area 308 as arange-finding point area on the AF sensor surface 302 corresponding toan area of the eyes.

Next, referring to FIGS. 4 to 6, an AF processing flow executed by theAE image processing unit 204 and the AF processing unit 205 according tothe present exemplary embodiment will be described.

FIG. 4 (including FIGS. 4A and 4B) illustrates a control flow of the AFprocessing unit 205. When an input of a switch (SW) 1 is receives by theoperation detection unit 213, the system control unit 203 controls theAF processing unit 205 to start AF processing. In step S401, the AFprocessing unit 205 makes a request to the AE image processing unit 204for performing various types of processing such as object detection,photometric calculation, and the like.

In step S402, the AF processing unit 205 starts charge storage drivingof the AF sensor. In step S403, the AF processing unit 205 waits fornotification from the AE image processing unit 204 of information of anobject to be tracked. Tracking processing executed by the AE imageprocessing unit 204 will be described below referring to FIG. 5. The AFprocessing unit 205 acquires, as information of the tracked object, anevaluation value of reliability indicating certainty of a detectionresult of the tracked object and positional information of the object onthe AF sensor surface.

In step S404, the AF processing unit 205 determines whether to adopt atracking result notified from the AE image processing unit 204. Thedetermination processing will be described below referring to FIG. 6. Instep S405, the AF processing unit 205 refers to the determination resultin step S404. If the AF processing unit 205 determines that the trackingresult is not adopted (NO in step S405), the processing proceeds to stepS410. If the AF processing unit 205 determines to adopt the trackingresult (YES in step S404), the processing proceeds to step S420.

In step S410, the AF processing unit waits for completion of chargestorage of a predetermined line sensor pair (e.g., center line sensorpair). Then, the AF processing unit 205 executes reading for a linesensor pair with respect to a range-finding point selected by the uservia the operation detection unit 213 and its surrounding range-findingpoints. The execution of reading around the selected range-finding pointis for capturing movement of the object in a horizontal or verticaldirection to the sensor surface to focus the camera. It is desirable toread sensor information within a range as wide as possible as long asprocessing time permits. Further, in an automatic selection mode forautomatically selecting an appropriate range-finding point from all therange-finding points, information is read from line sensor pairscorresponding to all the range-finding points.

In step S411, the AF processing unit 205 selects an optimal chargestorage result from among charge storage results of the line sensor pairread in step 410. More specifically, a charge storage result wherereliability of phase difference AF (reliability of image signal) ishigher than a predetermined threshold value is employed. The reliabilityof the phase difference AF is calculated from a degree of coincidencebetween two images, for example, as discussed in Japanese PatentApplication Laid-Open No. 2007-052072. Alternatively, when focusdetection is performed in a prediction mode, the AF processing unit 205selects, from past history, a sensor output result indicating an imagedeviation amount close to a prediction result as an optimal chargestorage result.

In step S412, the AF processing unit 205 notifies the AE imageprocessing unit 204 of position information of the range-finding pointcorresponding to the sensor position selected in step S411 and a requestfor updating a tracking target. For example, when an area 309 has beenselected as a range-finding point area illustrated in FIG. 3A, the AFprocessing unit 205 notifies the AE image processing unit 204 of an area310 illustrated in FIG. 3B corresponding to the area 309 on the AFsensor surface.

Processing in step S420 is performed when the tracking result notifiedfrom the AE image processing unit 204 is employed. The AF processingunit 205 determines whether charges have been stored in a line sensorpair (first signal generation unit) corresponding to the notifiedtracking range-finding point at the time when charge storage iscompleted for a predetermined line sensor pair (e.g., center line sensorpair).

There may be a plurality of range-finding points notified from the AEimage processing unit 204. In this case, all line sensor pairscorresponding to the plurality of range-finding points are targets fordetermining completion of charge storage. When charge storage has beencompleted for a certain target line sensor pair (YES in step S420), instep S430, the AF processing unit 205 executes reading for the linesensor pair corresponding to the tracking designated range-findingpoint.

For example, when the AE image processing unit 204 detects presence of atracking target in an area 312 illustrated in FIG. 3B, the AF processingunit 205 is notified of a corresponding area 313 on the AF sensorsurface 302 illustrated in FIG. 3A. In this step, the AF processing unit205 determines whether charge storage has been completed for all linesensor pairs corresponding to the area 313.

However, since the AE image processing unit 204 detects objects of samecolors as described below, as an object feature of the notified area313, a contrast component of a uniform color tends to be small. Thus,charge storage relatively takes time for a line sensor pair targeted fordetecting a contrast component. In step S430, reading of the line sensorpairs corresponding to the tracking designated range-finding point isexecuted. The AF processing unit 205 selects an optimal reading resultof the line sensor pair from among the reading results of the linesensor pairs.

The processing in step S421 is performed when charge storage is yet tobe completed for the line sensor pair corresponding to the range-findingpoint notified from the AE image processing unit 204 in step S420. Inthis step, the AF processing unit 205 determines whether to extend thecharge storage time of the sensor. In an embodiment of the presentinvention, the charge storage time is equivalent to the chargeaccumulation period. The determination processing will be described indetail below.

In step S422, the AF processing unit 205 refers to a determinationresult as to whether to extend the charge storage time of the linesensor pair corresponding to the tracking range-finding point executedin step S421. When the AF processing unit 205 determines not to extendthe charge storage time (NO in step S422), in step S424, the AFprocessing unit 205 cancels the charge storage of the AF sensor of thetracking range-finding point. Then, in step S430, the AF processing unit205 executes reading processing of the tracking range-finding point andselects an optimal charge storage result. When the AF processing unit205 determines to extend the charge storage time (YES in step S422), instep S423, the AF processing unit 205 waits for completion of the chargestorage of the AF sensor. Then, in step S430, the AF processing unit 205executes reading processing of the tracking range-finding point andselects an optimal charge storage result.

In step S431, the AF processing unit 205 refers to a range-findingresult of the tracking range-finding point selected in step S430 todetermine whether to adopt the tracking range-finding point. Forexample, when a defocus amount as a range-finding result is greatlydeviated compared with past range-finding history, it is determined thatthe tracking range-finding point is not adopted (NO in step S431), andthe processing proceeds to step S411. When adoption of the range-findingpoint is determined (YES in step S431), the processing proceeds to stepS432. In step S432, the AF processing unit 205 notifies the AE imageprocessing unit 204 of, among the range-finding points notified from theAE image processing unit 204, the range-finding point finally selectedin step S430, and the processing proceeds to step S440. The processingperformed by the AE image processing unit 204 will be described below.Based on the range-finding point information, the AF processing unit 205executes next tracking processing.

In step S440, the system control unit 203 instructs the lens MPU todrive a range ring of the lens based on a defocus amount derived fromthe charge storage result selected in step S411 or S430. The lensdriving unit 221 drives the range ring based on an instruction from thelens MPU.

In step S441, the system control unit 203 detects an input of anoperation of a SW 2. Then in step S442, the system control unit 203performs image capturing processing. If no operation input of the SW 2is detected (NO in step S441), the processing returns the AF processingin step S401 again.

Next, referring to a flowchart in FIG. 6, the determination as towhether to adopt the tracking result notified from the AE processingunit 204 in step S404 in FIG. 4 will be described.

In step S601, the system control unit 203 determines whether a trackingtarget changing request has been notified in step S412 of lastrange-finding processing. When the tracking target changing request hasbeen notified (YES instep S601), instep S603, the system control unit203 determines not to adopt the tracking result. When it is determinedthat no tracking target changing request has been notified instep S412of the last range-finding processing (NO in step S601), the processingproceeds to step S602.

In step S602, the system control unit 203 determines whether thetracking result notified from the AE image processing unit 204 isgreatly deviated on the sensor surface from the past range-findinghistory. When it is greatly deviated (YES in step S602), a possibilitythat the tracking result concerns detection of another object differentfrom the object to be tracked is high. Thus, the processing proceeds tostep S603, and the system control unit 203 determines not to adopt thetracking result. In this case, a threshold value of deviation on thesensor surface between the past selected range-finding point and therange-finding point of the tracking result is determined based on, forexample, an angle of view and an object distance. When none of theconditions is satisfied (NO in step S602), the processing proceeds tostep S604, and the system control unit 203 determines to adopt thetracking result.

Next, referring to a flowchart in FIG. 5, control performed by the AEimage processing unit 204 will be described. First, in step S501, the AEimage processing unit 204 waits to receive a processing request from theAF processing unit 205. When the processing request is received (YESinstep S501), the processing proceeds to step S502.

In step S502, the AE image processing unit 204 determines whether atracking target changing request has been notified from the AFprocessing unit 205. In the case of first tracking processing, the AEimage processing unit 204 determines that a tracking request has beenreceived. When it is determined that the tracking target changingrequest has been notified (YES in step S502), the processing proceeds tostep S503. In step S503, the AE image processing unit 204 sets an areafor detecting face or color feature information of the object based onthe range-finding point area selected by the AF processing unit 205 andnotified in step S412.

When it is determined that no tracking target changing request has beennotified (NO in step S502), the processing proceeds to step S504. The AEimage processing unit 204 sets, based on an area set in a last trackingarea, an area for detecting face or color feature information of theobject from its surrounding area. The area for detecting the featureinformation of the object is set around the last tracking area for thepurpose of capturing horizontal movement of the object on the sensorsurface. It is desirable to read sensor information of a range as wideas possible as long as processing time permits.

In step S505, the AE image processing unit 204 starts charge storagedriving of the AE sensor 211. In step S506, the AE image processing unit204 reads a charge storage result of the AE sensor 211.

In step S507, the AE image processing unit 204 determines whether a faceis present based on a reading result of the AE sensor 211 using a knownface detection technique. If a face is detected (YES in step S507), theprocessing proceeds to step S508, and whereas if not (NO in step S507),the processing proceeds to step S510.

In step S508, the AE image processing unit 204 determines presence orabsence of the tracking target changing request in step S502. If it isdetermined that the tracking target changing request has been made (YESin step S508), the processing proceeds to step S509. In step S509, theAE image processing unit 204 stores color information for colortracking. The color information is stored to be used for tracking by thecolor information of next frame and after. In this case, certainty of aface when the face is detected is calculated as tracking reliability.

If no face is detected (NO in step S507), in step S510, the AE imageprocessing unit 204 determines whether tracking can be performed basedon a color. In this case, the AE image processing unit 204 determinespresence or absence of the tracking target changing request in stepS502. When the tracking target changing request is present (YES in stepS510), the processing proceeds to step S511. Whereas the tracking targetchanging request is not present (NO in step S510), the processingproceeds to step S512.

In step S511, the AE image processing unit 204 stores the colorinformation of the object detected in the area set in step S503 as colorinformation for tracking. The color information is stored to be used fortracking by the color information of next frame and after. Thisprocessing corresponds to first tracking processing after a start of therange-finding, or tracking processing of next range-finding where the AFprocessing unit 205 does not adopt the tracking result.

In this case, for example, when the area 309 is selected as arange-finding point area illustrated in FIG. 3A, the AF processing unit205 notifies the AE image processing unit 204 of an area 310corresponding to the area 309. The color information stored by the AEimage processing unit 204 in this step is color information present inan AE division area 311 including the area 310. When a plurality ofcolors is present, the AE image processing unit 204 stores a color of alarge total area as a tracking color, and simultaneously stores shapesand sizes of areas of the same color.

In step S512, the AE image processing unit 204 searches for an areacoincident with the stored tracking color information in the detectiontarget area set in step S504. In this case, the AE image processing unit204 searches for an area where the color information of the detectiontarget area and the stored tracking color information coincide with eachother, and calculates a degree of the coincidence as an evaluation valuefor evaluating tracking reliability.

FIG. 10A illustrates a case where the AE image processing unit 204 isnotified of the area 310 on the AE sensor corresponding to an AFselection range-finding point, for example, as described above in stepS511. In FIG. 10A, the AE division area 311 including the area 310 is adetection target area. The AE image processing unit 204 cuts out a colorarea where a total area occupying areas determined as the same color inthe AE division area 311 is largest. FIG. 10A illustrates a case wherean area 320 indicated by diagonal lines in the AE division area 311 isdetermined as the largest area of the same color.

Then, in a next captured image, as illustrated in FIG. 10B, the AE imageprocessing unit 204 regards the area 320 indicated by the diagonal linesas a reference area, and searches for an area whose color informationcoincides with the reference area as a tracking area. During detectionof the tracking area, the reference area 320 is divided into a pluralityof areas. In the case illustrated in FIG. 10B, the reference area 320 isdivided into 10 areas.

The AE image processing unit 204 evaluates, for all the pixels 301 onthe AF sensor, a degree of coincidence of the color information betweeneach unit area 323 and the reference area 320. More specifically, the AEimage processing unit 204 divides the unit area 323 as in the case ofthe reference area 320, and compares color components between eachdivided area of the reference area 320 and each divided area of the unitarea 323. According to the present exemplary embodiment, a size of theunit area 323 is equal to that of the reference area 320, and the colorcomponents of the divided area in a corresponding position of both areasare compared with each other.

The AE image processing unit 204 calculates a result of adding togetherthe degrees of coincidence of the color components of the respectivedivided areas of the unit area 323 and the reference area 320 for allthe divided areas as an evaluation value of reliability, and sets a unitarea 323 having a highest evaluation value as a tracking area. Thedegrees of coincidence of the color components are evaluated by thisalgorithm to set a tracking target, and a degree of coincidence ofshapes is also evaluated as a result.

The AE image processing unit 204 lowers an evaluation value to reducereliability when a newly set tracking area is deviated from the lastdetermined tracking area on an x-y plane. This is in view of a highpossibility that the object is present in a place nearer to the lastdetected tracking area.

Thus, the AE image processing unit 204 selects, from among the targetareas, an area where color information best coincides with the trackingcolor information, and calculates a degree of coincidence as anevaluation value indicating reliability based on the stored shape andsize of the object.

In step S513, the AE image processing unit 204 notifies the AFprocessing unit 205 of tracking information. As the trackinginformation, information as to whether a face has been detected in stepS507, and pieces of information about the tracking range-finding areaand reliability acquired in steps S511 and S512 are included.

Next, referring to FIG. 7, AF charge storage time extensiondetermination processing of a tracking range-finding point according tothe first exemplary embodiment will be described. In step S421illustrated in FIG. 4, when charge storage is yet to be completed forthe line sensor pair corresponding to the range-finding point notifiedfrom the AE image processing unit 204, the AF processing unit 205determines whether to extend charge storage time of the sensor pair.

In step S701 illustrated in FIG. 7, the AF processing unit 205determines whether the tracking reliability notified from the AE imageprocessing unit 204 in step S513 illustrated in FIG. 5 is sufficientlyhigh. In other words, the AF processing unit 205 determines whether thetracking reliability is higher than a predetermined level. When thereliability is not higher than the predetermined level (NO in step S701,i.e., a second level), the processing proceeds to step S708. In stepS708, the AF processing unit 205 does not extend the charge storage timeand adopts a superior result from focus detection results irrespectiveof notification results from the AE image processing unit 204.

When the tracking reliability is higher than the predetermined level(YES in step S701, i.e., a first level), the processing proceeds to stepS702. In step S702, the AF processing unit 205 determines whether newlyacquired tracking positional information is not greatly deviated fromthe past history. For example, the AF processing unit 205 refers to pastphotographing history of several frames and, and even if a past trackingposition is near the area 309 illustrated in FIG. 3A, but the newlyacquired tracking position is greatly deviated as in the case of thearea 313 in FIG. 3A (YES in step S702), the processing proceeds to stepS708. If the newly acquired tracking position is not greatly deviatedfrom the past history, the processing proceeds to step S703 (NO in stepS702).

In step S708, the AF processing unit 205 adopts a superior result fromthe range-finding results without extending the charge storage time. Athreshold value of deviation in this case is determined based ondistance information of the object or a focal distance of the lens.

In step S703, the AF processing unit 205 determines whether the trackingresult notified from the AE image processing unit 204 is acquired bytracking based on the face information. If the tracking result is basedon the face information (YES in step S703), the processing proceeds tostep S704. Whereas if the tracking result is based on not the faceinformation but the color information (NO in step S703), the processingproceeds to step S707.

In step S704, the AF processing unit 205 determines whether an eyeposition has been detected in the face information of the trackingresult notified from the AE image processing unit 204. When it isdetermined that the eye position has been detected (YES in step S704),the processing proceeds to step S705. In step S705, the AF processingunit 205 controls driving of the AF sensor using first setting time asextended storage time.

When it is determined that no eye position has been detected (NO instepS704), instep S706, the AF processing unit 205 controls driving of theAF sensor using second setting time as the extended storage time. Instep S707, when the tracking result based on the color is detected, theAF processing unit 205 controls driving of the AF sensor using thirdsetting time as the extended storage time. The first to third settingtimes can individually be set. For example, the setting times can be setto satisfy, the first setting time<the second setting time<the thirdsetting. A reason for the settings is that when the eye position can beidentified, a contrast component can be detected relatively easily.

When a face is identified, it is expected that a contrast component canbe detected relatively easily compared with a case when a normal coloris detected. Thus, when an eye or a face is detected, releaseresponsiveness can be increased without setting the extended storagetime more than necessary according to the object.

On the other hand, in the tracking based on color information, asdescribed above in step S512, tracking reliability is determined to behigher as shapes or sizes are more similar with the same color.Accordingly, when tracking is performed based on the color information,detection of a contrast component tends to be difficult in the AFprocessing, and it is therefore desirable to secure longer chargestorage time. In this case, tracking reliability and detection accuracyof phase difference AF are set in inverse correlation. Thus, the thirdsetting time can be set by a reciprocal of an evaluation valueindicating the tracking reliability.

According to the present exemplary embodiment, the maximum permissiblestorage time of the AF sensor is set beforehand. To extend the chargestorage time, extended storage time is set so that total charge storagetime can be within the maximum permissible storage time.

When the tracking reliability is higher than a predetermined level, thecharge storage time can be extended by predetermined time, and when thetracking reliability is not higher than the predetermined level, thecharge storage time may not be extended. Further, in the case to set thecamera to more actively execute face detection, the first setting timeand the second setting time can be set longer.

As described above, in the present exemplary embodiment, when the phasedifference AF is performed corresponding to the area where the trackingbased on the face or the color information has been executed, theextended storage time of the AF sensor is set according to a feature ofthe object. Such control enables securing of range-finding accuracy ofthe phase difference AF while securing tracking performance.

Next, referring to FIG. 9, processing for waiting for completion of theAF charge storage when the charge storage time is extended in step S423illustrated in FIG. 4 will be described.

In step S901, the AF processing unit 205 waits for completion of chargestorage for one of the plurality of line sensor pairs corresponding tothe tracking area notified from the AE image processing unit 204.

In step S902, the AF processing unit 205 executes reading andcalculation processing for the charge storage completed line sensorpair, and acquires reliability of the phase difference AF as in the casein step S411.

In step S903, the AF processing unit 205 determines whether thereliability acquired in step S902 exceeds a predetermined thresholdvalue. When a charge storage result of any one of the line sensor pairsis sufficiently reliable (the reliability exceeds the predeterminedthreshold value, YES in step S903), the AF processing unit 205 cancelsdriving of the other line sensor pairs to end the current processing.

In other words, even when the charge storage time of the line sensorpair corresponding to the tracking range-finding point is extended, theAF processing unit 205 ends the charge storage when the reliability ofthe charge storage result of any one of the line sensor pairscorresponding to the tracking range-finding points acquired within theextended charge storage time is high. When the reliability of the chargestorage result of the read line sensor pair is not high (the reliabilitydoes not exceed the predetermined threshold value, NO in step S903), theprocessing proceeds to step S904.

In step S904, the AF processing unit 205 determines whether storedcharges of the line sensor pairs corresponding to all the trackingrange-finding points have been read. When there is still any line sensorpair to be read (NO in step S904), the processing returns to step S901.When reading of all the line sensor pairs has been completed (YES instep S904), the processing proceeds to step S905. The AF processing unit205 selects the best one from among the read results.

Through the processing described above, responsiveness of the AFprocessing can be secured by efficiently executing reading from the linesensor pair corresponding to the tracking area while adopting thetracking result of the AE image processing unit 204.

Next, a second exemplary embodiment will be described. Description ofparts similar to those of the first exemplary embodiment will beomitted. The second exemplary embodiment is different from the firstexemplary embodiment in charge storage time extension determinationprocessing of the line sensor pair corresponding to the trackingrange-finding point described above referring to FIG. 7. Hereinafter,referring to FIG. 8, the charge storage time extension determinationprocessing of a line sensor pair corresponding to a trackingrange-finding point according to the second exemplary embodiment will bedescribed.

In step S421 in FIG. 4, when the charge storage is yet to be completedfor the line sensor pair corresponding to the range-finding pointnotified from the AE image processing unit 204, the AF processing unit205 determines whether to extend the charge storage time of the linesensor pair. In the second exemplary embodiment, by using thedetermination in step S421 as a reference, the AF processing unit 205extends the charge storage time within a range where a continuousshooting speed set in the camera can be maintained. Accordingly, the AFprocessing unit 205 stores time of starting the AF processing in stepS401.

In step S801, the AF processing unit 205 determines whether the trackingreliability notified from the AE image processing unit 204 in step S513is sufficiently high. When the tracking reliability is low (NO in stepS801), the processing proceeds to step S807, and the AF processing unit205 selects a superior result from among the range-finding resultswithout extending the charge storage time. Whereas, when the trackingreliability is high (YES in step S801), the processing proceeds to stepS802.

In step S802, the AF processing unit 205 measures elapsed time T1 fromthe time stored in step S401 to step S421.

In step S803, the AF processing unit 205 calculates time T2 forreading/calculation for the line sensor pair corresponding to thetracking area notified from the AE image processing unit 204. Since thetracking area varies depending on detection results, the time 12 variesdepending on the number of line sensor pairs included in the trackingarea.

In step S804, the AF processing unit 205 calculates remaining time T3excluding the processing time T1 and T2 with respect to a condition ofthe continuous shooting speed set in the camera. In step S805, the AFprocessing unit determines whether there is spare in the time T3calculated in step S804. When it is determined that there is spare inthe time T3 with respect to the set continuous shooting speed (YES instep S805), then in step S806, the AF processing unit 205 sets extendedstorage time for the time T3. On the other hand, when there is no sparein the time T3 (NO in step S805), in step S807, the AF processing unit205 ends the processing without extending the storage time.

Through the processing described above, according to the presentexemplary embodiment, even when a tracking result by the AE imageprocessing unit 204 changes, the range-finding accuracy of the phasedifference AF corresponding to the tracking area can be secured whilemaintaining the continuous shooting speed set in the camera and securingtracking performance.

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or an MPU) that reads outand executes a program recorded on a memory device to perform thefunctions of the above-described embodiments, and by a method, the stepsof which are performed by a computer of a system or apparatus by, forexample, reading out and executing a program recorded on a memory deviceto perform the functions of the above-described embodiments. For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., non-transitory computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures, and functions.

This application claims priority from Japanese Patent Applications No.2011-226993 filed Oct. 14, 2011 and No. 2012-156628 filed Jul. 12, 2012,which are hereby incorporated by reference herein in their entirety.

What is claimed is:
 1. A focus adjustment apparatus comprising: anobject detection unit configured to detect feature information of anobject; a sensor unit provided with a plurality of sensor pairs each ofwhich accumulates charges to generate a pair of image signalsrespectively; and a focus detection unit configured to control chargeaccumulation time of the plurality of sensor pairs and detect a focusbased on a pair of image signals output from a first sensor pairscorresponding to the feature information of the object detected by theobject detection unit, wherein in a first case if the reliability of thedetection of the feature information of the object is a first levelwhich is higher than a second level, a charge accumulation time of thefirst sensor pairs is a first amount of time; wherein in a second caseif the reliability of the detection of the feature information is thesecond level, the charge accumulation time of the first sensor pairs isa second amount of time; and wherein the first amount of time is longerthan the second amount of time.
 2. The focus adjustment apparatusaccording to claim 1, wherein if the reliability of detection by theobject detection unit is the second level, after an end of chargeaccumulation of a predetermined sensor pair, the focus detection unitends the charge accumulation of the first sensor pair.
 3. The focusadjustment apparatus according to claim 1, wherein the object detectionunit tracks the object based on detected feature information.
 4. Thefocus adjustment apparatus according to claim 3, wherein if the objectdetection unit does not track an object, the focus detection unit endscharge accumulation of other sensor pairs after an end of chargeaccumulation of a predetermined sensor pair.
 5. The focus adjustmentapparatus according to claim 1, wherein if there is a plurality ofsensor pairs corresponding to the feature information of the object, thefocus detection unit detects a focus based on a pair of image signalsoutput from the sensor pair that corresponds to the feature informationwith the highest reliability.
 6. The focus adjustment apparatusaccording to claim 1, wherein if the reliability of the detection by theobject detection unit is the first level and if reliability of the imagesignals output from the first sensor pairs is higher than apredetermined threshold value within the set charge accumulation time,the focus detection unit ends the charge accumulation of the firstsensor pair.
 7. The focus adjustment apparatus according to claim 1,wherein the focus adjustment apparatus is configured to be included inan imaging apparatus, and wherein the charge accumulation time is setaccording to a continuous shooting speed of the imaging apparatus. 8.The focus adjustment apparatus according to claim 1, wherein the objectdetection unit detects color information as the feature information ofthe object.
 9. The focus adjustment apparatus according to claim 1,wherein the object detection unit detects face information as thefeature information of the object.
 10. The focus adjustment apparatusaccording to claim 9, wherein if face information is detected as thefeature information of the object, the focus detection unit sets thecharge accumulation time to be shorter than that when no faceinformation is detected as the feature information of the object. 11.The focus adjustment apparatus according to claim 9, wherein if the faceinformation is detected as the feature information of the object, theobject detection unit further detects eye information, and wherein ifthe eye information is detected, the focus detection unit sets thecharge accumulation time of the first sensor pairs to be shorter thanthat when no eye information is detected.
 12. The focus adjustmentapparatus according to claim 1, wherein the sensor pairs are linesensors each including a plurality of pixels.
 13. The focus adjustmentapparatus according to claim 1, wherein the focus adjustment apparatusis configured to be included in an imaging apparatus including an imagesensor configured to photoelectrically convert a light flux entered viaa photographic optical system, and wherein the sensor pairs are pixelson the image sensor.
 14. A method for controlling a focus adjustmentapparatus including a sensor unit provided with a plurality of sensorpairs each of which accumulates charges to generate a pair of imagesignals respectively, the method comprising: detecting featureinformation of an object; controlling charge accumulation time of theplurality of sensor pairs; and detecting a focus based on a pair ofimage signals output from a first sensor pair corresponding to thefeature information of the object, wherein in a first case if thereliability of the detection of the feature information of the object isa first level which is higher than a second level, a charge accumulationtime of the first sensor pairs is a first amount of time; wherein in asecond case if the reliability of the detection of the featureinformation is the second level, the charge accumulation time of thefirst sensor pairs is a second amount of time; and wherein the firstamount of time is longer than the second amount of time.
 15. The focusadjustment apparatus according to claim 1, wherein the focus detectionunit is configured to detect the focus based on the phase differencebetween each of the pair of image signals.
 16. An imaging apparatuscomprising: an image sensor; an object detection unit configured todetect feature information of an object and output an evaluation valueof a reliability of the detection of the feature information of theobject; and a focus detection unit configured to detect a focus basedupon autofocus signals which represent the focus at a plurality ofpoints which correspond to the feature information of the object;wherein a charge accumulation time of a plurality of sensors used togenerate the autofocus signals is increased above a default chargeaccumulation time if the reliability of the detection of the featureinformation of the object is above a threshold.